39 research outputs found
ResViT: Residual vision transformers for multi-modal medical image synthesis
Multi-modal imaging is a key healthcare technology that is often
underutilized due to costs associated with multiple separate scans. This
limitation yields the need for synthesis of unacquired modalities from the
subset of available modalities. In recent years, generative adversarial network
(GAN) models with superior depiction of structural details have been
established as state-of-the-art in numerous medical image synthesis tasks. GANs
are characteristically based on convolutional neural network (CNN) backbones
that perform local processing with compact filters. This inductive bias in turn
compromises learning of contextual features. Here, we propose a novel
generative adversarial approach for medical image synthesis, ResViT, to combine
local precision of convolution operators with contextual sensitivity of vision
transformers. ResViT employs a central bottleneck comprising novel aggregated
residual transformer (ART) blocks that synergistically combine convolutional
and transformer modules. Comprehensive demonstrations are performed for
synthesizing missing sequences in multi-contrast MRI, and CT images from MRI.
Our results indicate superiority of ResViT against competing methods in terms
of qualitative observations and quantitative metrics
HydraViT: Adaptive Multi-Branch Transformer for Multi-Label Disease Classification from Chest X-ray Images
Chest X-ray is an essential diagnostic tool in the identification of chest
diseases given its high sensitivity to pathological abnormalities in the lungs.
However, image-driven diagnosis is still challenging due to heterogeneity in
size and location of pathology, as well as visual similarities and
co-occurrence of separate pathology. Since disease-related regions often occupy
a relatively small portion of diagnostic images, classification models based on
traditional convolutional neural networks (CNNs) are adversely affected given
their locality bias. While CNNs were previously augmented with attention maps
or spatial masks to guide focus on potentially critical regions, learning
localization guidance under heterogeneity in the spatial distribution of
pathology is challenging. To improve multi-label classification performance,
here we propose a novel method, HydraViT, that synergistically combines a
transformer backbone with a multi-branch output module with learned weighting.
The transformer backbone enhances sensitivity to long-range context in X-ray
images, while using the self-attention mechanism to adaptively focus on
task-critical regions. The multi-branch output module dedicates an independent
branch to each disease label to attain robust learning across separate disease
classes, along with an aggregated branch across labels to maintain sensitivity
to co-occurrence relationships among pathology. Experiments demonstrate that,
on average, HydraViT outperforms competing attention-guided methods by 1.2%,
region-guided methods by 1.4%, and semantic-guided methods by 1.0% in
multi-label classification performance
Simultaneous use of Individual and Joint Regularization Terms in Compressive Sensing: Joint Reconstruction of Multi-Channel Multi-Contrast MRI Acquisitions
Purpose: A time-efficient strategy to acquire high-quality multi-contrast
images is to reconstruct undersampled data with joint regularization terms that
leverage common information across contrasts. However, these terms can cause
leakage of uncommon features among contrasts, compromising diagnostic utility.
The goal of this study is to develop a compressive sensing method for
multi-channel multi-contrast magnetic resonance imaging (MRI) that optimally
utilizes shared information while preventing feature leakage.
Theory: Joint regularization terms group sparsity and colour total variation
are used to exploit common features across images while individual sparsity and
total variation are also used to prevent leakage of distinct features across
contrasts. The multi-channel multi-contrast reconstruction problem is solved
via a fast algorithm based on Alternating Direction Method of Multipliers.
Methods: The proposed method is compared against using only individual and
only joint regularization terms in reconstruction. Comparisons were performed
on single-channel simulated and multi-channel in-vivo datasets in terms of
reconstruction quality and neuroradiologist reader scores.
Results: The proposed method demonstrates rapid convergence and improved
image quality for both simulated and in-vivo datasets. Furthermore, while
reconstructions that solely use joint regularization terms are prone to
leakage-of-features, the proposed method reliably avoids leakage via
simultaneous use of joint and individual terms.
Conclusion: The proposed compressive sensing method performs fast
reconstruction of multi-channel multi-contrast MRI data with improved image
quality. It offers reliability against feature leakage in joint
reconstructions, thereby holding great promise for clinical use.Comment: 13 pages, 13 figures. Submitted for possible publicatio
CalibFPA: A Focal Plane Array Imaging System based on Online Deep-Learning Calibration
Compressive focal plane arrays (FPA) enable cost-effective high-resolution
(HR) imaging by acquisition of several multiplexed measurements on a
low-resolution (LR) sensor. Multiplexed encoding of the visual scene is
typically performed via electronically controllable spatial light modulators
(SLM). An HR image is then reconstructed from the encoded measurements by
solving an inverse problem that involves the forward model of the imaging
system. To capture system non-idealities such as optical aberrations, a
mainstream approach is to conduct an offline calibration scan to measure the
system response for a point source at each spatial location on the imaging
grid. However, it is challenging to run calibration scans when using structured
SLMs as they cannot encode individual grid locations. In this study, we propose
a novel compressive FPA system based on online deep-learning calibration of
multiplexed LR measurements (CalibFPA). We introduce a piezo-stage that
locomotes a pre-printed fixed coded aperture. A deep neural network is then
leveraged to correct for the influences of system non-idealities in multiplexed
measurements without the need for offline calibration scans. Finally, a deep
plug-and-play algorithm is used to reconstruct images from corrected
measurements. On simulated and experimental datasets, we demonstrate that
CalibFPA outperforms state-of-the-art compressive FPA methods. We also report
analyses to validate the design elements in CalibFPA and assess computational
complexity